Skip to content

Conversation

leontyevdv
Copy link
Contributor

Add a new ES|QL function that filters @timestamp values for the given time range.

TRANGE(<start time>, <end time>)
TRANGE(<time interval>)

Filters @timestamp values for the given time range. The first version implicitly uses now() as end time, giving an interval for the last hour or day, while the second one can consume regular DateTime specs or negative intervals:

TRANGE(1h) is equivalent to @timestamp >= now() - 1 hour

TRANGE(-3h, -2h) is equivalent to @timestamp >= now() - 3 hour AND @timestamp < now() - 2h

TRANGE(2024-05-12T12:00:00, 2024-05-12T15:30:00) is equivalent to @timestamp >= DATE_PARSE(2024-05-12T12:00:00) AND @timestamp < DATE_PARSE(2024-05-12T15:30:00)

TRANGE(1715504400000, 1715517000000) is equivalent to the above, using ms since Epoch instead of text-formatted dates.

Closes #135599

Add a new ES|QL function that filters @timestamp values for the given
time range.

Closes elastic#135599
@leontyevdv leontyevdv requested a review from dnhatn October 11, 2025 00:02
@leontyevdv leontyevdv self-assigned this Oct 11, 2025
@leontyevdv leontyevdv added >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) :StorageEngine/TSDB You know, for Metrics :Analytics/ES|QL AKA ESQL Team:StorageEngine :StorageEngine/ES|QL Timeseries / metrics / logsdb capabilities in ES|QL labels Oct 11, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@elasticsearchmachine
Copy link
Collaborator

Hi @leontyevdv, I've created a changelog YAML for you.

Copy link
Contributor

github-actions bot commented Oct 11, 2025

🔍 Preview links for changed docs

Copy link
Contributor

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

@leontyevdv leontyevdv requested a review from a team October 11, 2025 00:10
Add a new ES|QL function that filters @timestamp values for the given
time range.

Closes elastic#135599
@dnhatn
Copy link
Member

dnhatn commented Oct 12, 2025

Thank you, Dima! Is there a reason we don't translate TRANGE to GreaterThanEquals() or BinaryLogicOperation(GreaterThanEquals, LessThanOrEquals)?

@leontyevdv
Copy link
Contributor Author

Thank you, Dima! Is there a reason we don't translate TRANGE to GreaterThanEquals() or BinaryLogicOperation(GreaterThanEquals, LessThanOrEquals)?

Hi @dnhatn ! Thank you for the review! Initially, I implemented TRANGE as a surrogate and struggled with the tests for quite a while. The issue was with the assertion ("Duplicate name ids are not allowed in layouts"): see Layout. Then, inspired by the StartsWith function, I rewrote it using TranslationAware.SingleValueTranslationAware directly.

I have the surrogate version in my stash. We can take a look at it together. Wdyt?

# Conflicts:
#	x-pack/plugin/esql/qa/testFixtures/src/main/resources/k8s-timeseries-rate.csv-spec
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java
Replace asQuery by a surrogate

Closes elastic#135599
Replace asQuery by a surrogate

Closes elastic#135599
Fix constructor visibility

Closes elastic#135599
Copy link
Member

@dnhatn dnhatn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Dima! I left some comments, but the PR looks promising.

I have a few use cases we might need to discuss:

  1. Modifier: Limit time range to one hour from "2025-01-04T07:40:00+00:00"?

  2. Offset: Limit time range to 1 hour a week from now: TRANGE(1w + 1h, 1w). I'm not sure if this is valid syntax. Can we provide a better syntax?

  3. Start only: TRANGE("2025-01-04T07:40:00+00:00"), which means TRANGE("2025-01-04T07:40:00+00:00", now)?


resolution = isType(
startTime,
dt -> dt == KEYWORD || dt == LONG || dt == INTEGER,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why we allow longs and ints here?

if (endTime == null) {
Object start = startTime.fold(foldContext);
if (start == null) {
throw new InvalidArgumentException("start_time_or_interval can not be null in [{}]", sourceText());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we prefer reporting this in PostAnalysisVerificationAware. Can you implement PostAnalysisVerificationAware for this class?

@dnhatn dnhatn self-requested a review October 15, 2025 16:17
@kkrik-es
Copy link
Contributor

  1. Offset: Limit time range to 1 hour a week from now: TRANGE(1w + 1h, 1w). I'm not sure if this is valid syntax. Can we provide a better syntax?

This one is interesting.. I'd say we leave this for later, as the semantics are more elaborate. @dnhatn wdyt?

The rest should be fine.

@leontyevdv
Copy link
Contributor Author

leontyevdv commented Oct 16, 2025

  1. Offset: Limit time range to 1 hour a week from now: TRANGE(1w + 1h, 1w). I'm not sure if this is valid syntax. Can we provide a better syntax?

This one is interesting.. I'd say we leave this for later, as the semantics are more elaborate. @dnhatn wdyt?

The rest should be fine.

Thanks, folks! To confirm what I'm going to add and what have been implemented already:

[DONE] TRANGE(2024-05-12T12:00:00, 2024-05-12T15:30:00) - [explicit start; explicit end]
[DONE] TRANGE(1715504400000, 1715517000000) - [explicit start in millis; explicit end in millis]
[DONE] TRANGE(1h) - [now - 1h; now] - supports time_duration (1h, 1min, etc) and period (1day, 1month, etc.)

Explicit start time until Now:
[TODO] TRANGE(2024-05-12T12:00:00) - [explicit start; now]
[TODO] TRANGE(1715504400000) - [explicit start in millis; now]

Modifier for an explicit start time:
[TODO] TRANGE(2024-05-12T12:00:00, 1h) - [explicit start; start + 1h] - supports time_duration (1h, 1min, etc) and period (1day, 1month, etc.)
[TODO] TRANGE(2024-05-12T12:00:00, -1h) - [explicit start; start - 1h] - supports time_duration (1h, 1min, etc) and period (1day, 1month, etc.)

Does this make sense?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL >enhancement :StorageEngine/ES|QL Timeseries / metrics / logsdb capabilities in ES|QL :StorageEngine/TSDB You know, for Metrics Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:StorageEngine v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ES|QL: Add TRANGE function

4 participants